The application relates generally to multimedia encoders and specifically an integrated multimedia stream multiplexer.
Multimedia encoders, such as those used for MPEG and MPEG2 encoding, provide the necessary compression to allow video and audio data to be transferred, stored, and played in a computer environment. Conventional MPEG encoders typically use two encoders, video and audio, which receive data transmitted from a data source. Each encoder is coupled to a separate memory for storing the video and audio data. The video encoder compresses the video data and transmits the compressed data to the conventional stream multiplexer controller. The audio encoder performs the same tasks and transmits the compressed audio data to the stream multiplexer controller. In an MPEG2 environment, the two elementary streams are multiplexed by the stream multiplexer controller to generate either a Program or Transport stream, depending on the embodiment. The Program or Transport stream are stored in a separate PS/TS memory, or are fed directly to another device, such as a communications link. In this conventional system, the stream multiplexer operates independently of the video and audio encoders, and therefore the output bit rate of the Program or Transport stream is unpredictable, as it will vary as the bit rates of the elementary streams vary. The unpredictable bit rate of the output stream makes it more difficult to process the Program or Transport stream downstream. Additionally, in conventional systems when the bit rate of one stream decreases, the bit rate of the Program or Transport stream may decrease correspondingly, thus lowering the system throughput unnecessarily. The use of independent memory for each encoder also leads to underutilization of system resources. When the bit rate of a stream decreases, less data is stored in the corresponding memory by the corresponding encoder. However, in conventional systems, the newly available storage space remains unused until the bit rate for the stream increases. Finally, conventional stream processors process incoming elementary streams using a predetermined ratio-based approach which restricts the granularity of the multiplexing as the ratios are fixed and cannot be optimized for the requirements of different output streams.
Therefore, an integrated multimedia encoding system which is capable of generating a program or transport stream at a bit rate at or near the maximum bit rate capacity for the system is needed. Additionally, an integrated multimedia encoding system which operates with reduced memory storage requirements is also needed. Finally, a programmable elementary stream multiplexer is needed which takes advantage of dedicated instructions to perform the task of generating a single output data stream, and which is flexible to adjust the data rate for different formats.
In accordance with the present invention, an integrated multimedia encoding system is disclosed. Multimedia encoders which are capable of adjusting bit rates receive multimedia data to compress the data. After compressing the data, the multimedia encoders adjust the bit rates of the elementary streams responsive to a control input. Bit rates may be increased or decreased using delays, and generating more or less bits for each macroblock, frame or group of frames, or by increasing or decreasing the rate at which multimedia data is transmitted to the encoders. A unified memory module is coupled to the multimedia encoders to store the multimedia elementary stream data, the Program or Transport stream data, and data from other sources as needed. The unified memory is capable of adjusting storage allocations responsive to the realtime requirements of the incoming multimedia streams and the outgoing Program or Transport stream data. A stream processor is coupled to the unified memory module and the multimedia encoders for multiplexing the elementary streams into a single stream, and monitoring the actual bit rate of the combined multimedia stream. Monitoring the actual bit rate as a function of number of bits passed over a period of time provides accurate feedback as to the system throughput. A multimedia processor then determines the bit rates of the elementary streams, and generating a control signal to adjust the bit rates of the encoder to ensure that an optimal bit rate is continuously achieved by the system. The stream processor also operates using dedicated instructions which allow the stream processor to efficiently multiplex the incoming streams together.